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(57) Abstract 

In order to remove blocking artefacts from a frame which has been coded by blocks and then decoded, a certain number of pixels 
(n) is selected for examination from both sides of the block boundary (30). The number of pixels selected for examination depends on the 
image content of the frame in the environment of the block boundary, particularly on the difference of the pixel values across the block 
boundary (30) and the size of the quantization step of the transformation coefficients used in the transformation coding of the blocks. 
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Adaptive filter 



The invention relates to the processing of a digital video image in general. In 
particular, the invention relates to removing image errors caused by block 
5 boundaries in connection with the decoding of a coded video image. 

An arrangement like the one shown in Figure 1 is generally used for transferring a 
digital video image in compressed form. The video image is formed of sequential 
frames. The current frame comes to the transmission system 1 0 as input data l n (x,y). 
In the differential summer 11 it is transformed into a differential frame E n (x > y) by 

10 subtracting from it a prediction frame P n (x,y) formed on the basis of previous 
images. The differential frame is coded in block 12 in the manner described 
hereinafter, and the coded differential frame is directed to the multiplexer 13. For 
forming a new prediction frame, the coded differential frame is also directed to the 
decoder 14, which produces a decoded differential frame E n (x,y) which is summed 

15 in the summer 15 with the prediction frame P n (x,y), resulting in a decoded frame 
l„(x,y ). It is saved in the frame memory 1 6. For coding the next frame, the frame 
saved in the frame memory is read as a reference frame R n (x,y) and in the motion 
compensation and prediction block 17 it is transformed into a new prediction frame 
according to the formula 

20 P n (x,y) = R n [x + Ax(x,y), y + Ay(x,y)] ( 1 ) 

The pair of numbers [Ax(x,y), Ay(x,y)] is called the motion vector of the pixel at 
location (x,y) and the numbers Ax(x,y) and Ay(x,y) are the horizontal and vertical 
shifts of the pixel. They are calculated in the motion estimation block 18 and the 
motion information coding block 19.The set of motion vectors (Ax(), Ay( )] 

25 consisting of all motion vectors related to the pixels of the frame to be compressed 
is also directed to the multiplexer 13, which multiplexes it into the same data stream 
with a coded differential frame for sending to a receiver. Some frames can be partly 
or entirely so difficult to predict that it is not practical to use prediction when 
coding them. These frames or parts of frames are coded using so-called intracoding 

30 without prediction, and motion vector information relating to them is not sent to the 
receiver. 

In the receiver 20, the demultiplexer 21 separates the coded differential frames and 
the motion information transmitted by the motion vectors and directs the coded 
differential frames to the decoder 22, which produces a decoded differential frame 
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E n (x,y), which is summed in the summer 23 with the prediction frame P„(x,y) 
formed on the basis of previous frames, resulting in a decoded frame I„(x,y). It is 
directed to the output 24 of the reception decoder and at the same time saved in the 
frame memory 25. For decoding the next frame, the frame saved in the frame 
5 memory is read as a reference frame and transformed into a new prediction frame in 
the motion compensation and prediction block 26, according to the formula (1) 
presented above. 

The coding method which is applied in block 12 to the coding of the differential 
frame or to the intracoding of a frame or part of it to be sent without prediction, is 

10 generally based on a transformation, the most common of which is Discrete Cosine 
Transformation, DCT. The frame is divided into adjacent blocks sized e.g. 8x8 
pixels. In coding and decoding, the blocks are processed independent of one 
another. The transformation is calculated for the block to be coded, resulting in a 
series of terms dependent on a certain variable. The coefficients of these terms are 

15 quantized on a discrete scale in order that they can be processed digitally. 
Quantization causes rounding errors, which can become visible in an image 
reconstructed from blocks so that there is a discontinuity of pixel values at the 
boundary of two adjacent blocks. Because a certain decoded frame is used for 
calculating the prediction frame for the next frames, the errors can be repeated in 

20 sequential frames, thus causing visible edges in the image reproduced by the 
receiver. Image errors like this are called blocking artefacts. 

Some prior art methods are known for removing the blocking artefacts. These 
methods are characterized by the following features: 

- determining which pixel requires value correction for removing the blocking 
25 artefact, 

- determining a suitable low-pass filtering for each pixel to be corrected, based on 
the values of other pixels contained by the filtering window placed around the pixel, 

- calculating a new value for the pixel to be corrected, and 

- rounding the new value to the closest digitized tonal value. 

30 Factors that influence the selection of a filter and a decision to introduce a filter can 
be, for example, the difference between the values of pixels across the block 
boundary, the size of the quantization step of the coefficients received as the 



WO 98/41025 



PCT/FI98/00219 



3 

transformation result, and the difference of the pixel values on different sides of the 
pixel being processed. 

It has been found that the prior art methods also tend to remove lines of the image 
that should really be part of it. On the other hand, the prior art methods are not 
5 always capable of removing all blocking artefacts. 

The objective of the present invention is to present a new kind of filtering 
arrangement for removing the blocking artefacts. The invention also has the 
objective that the method and device according to it operate more reliably and 
efficiently than the prior art solutions. 

10 The objectives of the invention are achieved by adapting the selection of pixels for 
filtering and the filtering process more flexibly than before to the features of the 
frame and the environment of the filtering point. 

The method according to the invention for removing blocking artefacts from a frame 
that has been coded by blocks and decoded after that, in which method a certain 
15 number of pixels from both sides of the block boundary are selected for 
examination, is characterized in that the number of pixels selected for examination 
depends on the image content of the frame in the environment of the block 
boundary. 

The invention also relates to a device for implementing the method according to the 
20 invention. The device according to the invention is characterized in that it comprises 
means that operate adaptively according to the image content of the frame for 
selecting the pixels for examination and filtering. 

Because blocking artefacts only occur at block boundaries, the filtering according to 
the invention is only focused on pixels at block boundaries and the immediate 
25 vicinity thereof. Edges that are part of the image can be anywhere in the image area. 
In order that only pixels containing blocking artefacts would be selected for 
correction and that the quality of edges that are part of the image would not be 
deteriorated during filtering, the following assumptions have been made in the study 
that resulted in the invention: 



30 - in edges that are part of the image, the tonal value change is generally larger than 
in blocking artefacts, and 
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- the real image edges in which the tonal value change is small do not suffer 
considerably from the rounding of the tonal value difference by filtering. 

Because the image to be coded is generally divided into blocks both vertically and 
horizontally, the image contains both vertical and horizontal block boundaries. With 

5 regard to vertical block boundaries, there are pixels to the right and left of the 
boundary, and with regard to the horizontal block boundary, there are pixels above 
and below the boundary. In general, the location of the pixels can be described as 
being on the first and second side of the block boundary. In the filtering according 
to the invention, the number of pixels to be corrected, the characteristic features of 

10 the filter being used and the size of the filtering window depend on the following 
factors: 

a) the difference of the pixel values A across the block boundaiy (the difference 
can be defined in many ways; one definition is A = |rj - 1||, where n is the value of 
the pixel on the first side of the block boundary closest to the boundary, and l t is the 

15 value of the pixel on the second side of the block boundaiy closest to the boundary), 

b) the size of the quantization step QP of the coefficients received as the result of 
the transformation used in the coding, and 

c) the differences of the pixel values between the pixels on the first side of the 
block boundary, and correspondingly between the pixels on the second side of the 

20 block boundary. 

In the method and device according to the invention, the number of the pixels to be 
selected for filtering can vary, and it is not necessarily the same on different sides of 
the block boundary. Because the number of pixels is adapted according to the 
factors mentioned above to the general features of the image information contained 
25 by the frame in the domain in question, the method is more reliable than the prior 
art methods. A larger number of blocking artefacts can be removed without 
weakening the real image edges unreasonably. 

In the following, the invention will be described in more detail with reference to the 
preferred embodiments and the accompanying drawings, in which 

30 Figure 1 represents a prior art video image transfer arrangement, 

Figure 2 represents the location of pixels in relation to the block boundary in the 
method according to the invention, 
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Figure 3 represents alternatives for locating the filtering according to the invention 
in a video image transfer arrangement, 

Figure 4 represents schematically a device for implementing a method according 
to the invention, and . 

5 Figure 5 represents a device according to Figure 4 in operation. 

In the above, in connection with the description of the prior art, reference was made 
to Figure 1, and so in the following description of the invention and its preferred 
embodiments, reference will be made mostly to Figures 2 to 5. The same reference 
numbers are used for corresponding parts in the figures. 

10 Figure 2 shows the location of the pixels ri - r 6 and \\ - 1 6 in relation to the vertical 
block boundary 30. For implementing the method according to the invention, certain 
parameters must be specified at first. The parameter n is the largest number of pixels 
to be examined from the block boundary to one direction, and its value is 6 in the 
case of Figure 2. It is practical to select the value of the parameter n so that it has a 

15 certain relation to both the difference of the pixel values A across the block 
boundary and to the size of the quantization step QP of the coefficients received as 
the result of the coding. The following definition is recommended for use here: 

0 A > 2.00a 

1 1.50a < A < 2.00a 

2 1.00a < A < 1.50a 
w = < 3 0.66a < A < 1 00a , (2) 

4 0.40a < A <066a 

5 0.25a < A <040a 

6 0< A <025a 

wherein a = QPlog(QP); If QP has a different value in blocks on different sides of 
20 the block boundary, the smaller value of QP is used in calculation, as well as in all 
cases presented hereinafter, in which a definition includes reference to one QP value 
only. The invention does not limit the determination of the value of parameter n, but 
according to the guidelines of the definition (2) it is advantageous that its value is 
generally higher when the difference of pixel values A across the block boundary is 
25 small in comparison to the size of the quantization step QP of the coefficients 
received as the result of the coding transformation. If the difference between the 
pixel values A is very large, there is probably a real image edge at the block 
boundary, and the pixels are not examined at this point for filtering at all (n=0). 
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The next step is to determine the values of the parameters d t and d r , which represent 
activity, or the differences of pixel values betweel pixels on one side of the block 
boundary. For the parameter d rj one preferred definition is the following: 

d r = 6, if |r, - rj| < p/j with allje[l,6], 
5 otherwise: d r = i, where i must meet the conditions 

ie[l,n], (3) 

In - rj.il > p/i, and 

|r i - rj| < p/j with allje[l,i]. 

Here the auxiliary parameter P = 41og(QP). The value of the parameter d| is 
10 determined similarly, except that all r's are replaced by I's. The number 6 occurring 
in the definition (3) is the result of the fact that the highest possible value of n is 6 
according to the definition (2). If n is defined differently, but the parameters d r and 
dj are defined according to definition (3), the number 6 must be replaced by the 
highest possible value of n according to the new definition. 

15 With regard to the invention, it is advantageous that the values of the parameters d r 
and d| are calculated independent of one another, because the image information 
contained by the frame can be different on different sides of the block boundary. 
The invention does not limit the definition of parameters d r and d|, but according to 
the guidelines of the definition (3) it is advantageous that these parameters arc used 

20 to limit the blocking artefact processing relatively close to the block boundary, if 
there is a real image edge beside the block boundary. The essential features of the 
definition (3) can be described so that the value of parameter d, (and 
correspondingly the value of parameter d|) tells how many pixels counted from the 
block edge have approximately the same value as the pixel at the block edge. 

25 A high value of parameter n (e.g. 6) tells that the difference between the pixel 
values at the block boundary is relatively small as compared to the general variation 
of the pixel values in the block area. In this case, it is possible that there is a real 
image edge just beside the block boundaiy. By selecting a sufficiently small value 
of parameter d r (or d|), it is possible to restrict the filtering aimed at correcting 

30 blocking artefacts so that it does not have a deteriorating effect on a real image edge 
close to the block boundaiy. In some situations, a large number of pixels counted 
from the block edge have approximately the same value as the pixel at the block 
edge. In that case, the definition (3) would give the parameter d r or (d|) a relatively 
high value, but if there is a clear leap of pixel values between the blocks, the small 

35 value of the parameter n due to this and the reference to the value of n in the 
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definition (3) make sure that an unreasonably high value is not selected as the value 
of the parameter d r (or d|), which would result in unnecessary filtering. 

In addition, the largest possible number of pixels to be filtered must be decided. 
This does not have a notation of its own in Figure 2, but it can be e.g. 3, which 
5 means that filtering can only be used to correct the value of the pixels r u r 2 , \\ 1|, 1 2 
and 1 3 . 

When the values of the parameters n, d r and d| have been determined, filtering is 
carried out using a suitable filter. The invention does not set limits to the kind of 
filter to be used, but a filtering arrangement that has been found preferable, will be 

10 described in the following. Filtering is used to determine a new value for the pixels 
selected for filtering. For the new value of the pixel being filtered at a given instant, 
the mean value of the pixels that appear in the filtering window is calculated. In this 
case, the filtering window is symmetrical with regard to the pixel to be filtered and 
contains, in addition to die pixel to be filtered, one, two or three pixels from its both 

15 sides, depending on the values of the parameters d r and d| as described hereinafter. 
The calculated mean value is rounded to the closest digitized tonal value, whereby it 
becomes the new value of the filtered pixel. 

The following table shows the determination of the width of the filtering window 
for the pixels r u r 2 and r 3 according to the value of the parameter d f . The values of 

20 the pixels 1|, 1 2 and I3 are determined in the same manner according to the value of 
the parameter d». In the table, X means that the pixel in question is not filtered at all, 
and the number means that the filtering window includes a quantity of pixels shown 
by the number from each side of the pixel being examined. Among other things, the 
table shows that for the filtering to be focused on any pixel, the parameters d f and d) 

25 must both have a value higher than 1 . 
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d r (d,> 1) 


ri 


T2 


1"3 


1 


X 


X 


X 


2 


1 


X 


X 


3 


1 


1* 


X 


4 


2 


2 


X 


5 


2 


2 


2** 


6 


3 or 2*** 


3 


3 



* the filtered value of pixel r, is used for filtering of pixel r 2 

** the filtered values of pixels ri and r 2 are used for filtering the pixel r 3 

*** 3 if di> 2, otherwise 2. 

5 The above description relates to implementing the filtering on one horizontal part of 
a pixel row, which part is 12 pixels long and located symmetrically on both sides of 
a vertical block boundary. The description can be easily generalized to concern 
vertical parts of pixel columns, which are located symmetrically on both sides of a 
horizontal block boundary: Figure 2 can be turned 90 degrees counter-clockwise, 

10 whereby the block boundary 30 is horizontal, and the pixels shown in the figure 
form part of the vertical pixel column so that pixels i ( - r (> are the pixels above and 
pixels 1| - 1(, are the pixels below. For filtering the whole frame with the method 
according to the invention, all vertical block boundaries of the frame are examined 
row by row and all its horizontal block boundaries column by column. The order 

15 has no significance at such, and thus all the horizontal block boundaries of the 
frame could be examined first column by column, and then all the vertical block 
boundaries row by row. 

Figure 3 shows at which points the prior art image transfer arrangement can be 
improved by filtering according to the invention. The first alternative is to place the 

20 block implementing the filtering according to the invention in the output of the 
decoder of the receiver as illustrated by reference number 31. In this case, only the 
video image being directed out during decoding is filtered. Another alternative is to 
place the block carrying out the filtering according to the invention in the receiver 
before the point at which the decoded frame is directed to the frame memory 25 for 

25 forming a prediction frame, as illustrated by reference number 32. This alternative 
has the advantage that the removal of blocking artefacts also has an effect on the 
formation of a prediction frame, whereby the lines possibly caused by blocking 
artefacts and seen by the naked eye are not duplicated via the prediction frame to 
the next frames. In order to achieve the last mentioned effect, the block carrying out 
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the filtering according to the invention can also be placed before the frame memory 
25 or after it, but the location shown by reference number 32 is preferred, because 
when done at this stage, the filtering influences simultaneously the frame to be 
directed out and the frame to be saved in the memory. In the transmitter, the block 
5 carrying out the filtering according to the invention can be placed as shown by 
reference numbers 33 and 34 before the frame memory 16 or after it, if it is desired 
that the invention is also applied to producing a corrected prediction frame at the 
transmission end. 

The block carrying out the filtering according to the invention is particularly 
10 advantageously implemented in a digital signal prosessor or a corresponding device 
suited for processing a digital signal, which can be programmed to focus 
predetermined processing functions to the signal received as input data. At the 
programming stage of the digital signal processor, the definitions 35 - 38 are saved 
according to Figure 4 for calculating the parameters that control filtering. At the 
1 5 operation step according to Figure 5, the frame is saved temporarily in the register 
40, so that it can be processed in the signal processor pixel by pixel. A number of 
pixels indicated by the parameter n are selected from the frame as the pixels to be 
examined 4 1 at a given instant from each side of a certain point of a certain block 
boundary, the d-parameters 42 are calculated, filtering 43 is done, and these 
20 measures are repeated, until all boundaries of all blocks have been gone through, 
after which the frame can be fed out of the register 40 and a new frame saved for 
processing. The measures according to Figures 4 and 5 can be carried out in a 
separate signal processor or they can be part of the operation of such a signal 
processor which also contains other arrangements for signal processing. 

25 The invention can be modified without departing from the scope defined by the 
claims hereinafter, using the capabilities of a person skilled in the art without actual 
inventive steps. For example, the parameter A can be calculated using the formula A 
= I0"i + *2)-(li + h)\ or some other formula regarded as suitable. The definitions of 
other parameters above have also been intended as examples only. A particularly 

30 advantageous use of the invention is in digital television receivers and other devices 
that receive and decode digital video image. 
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Claims 

1. A method for removing blocking artefacts from the frame of a digital video 
signal, which has been coded by blocks and then decoded, in which method a 
certain number of pixels (n) are selected for examination from both sides of the 

5 block boundary (30), and the value of the pixels selected for examination is 
corrected by filtering, characterized in that the number of pixels (n) selected for 
examination depends on the image content of the frame in the environment of the 
block boundary (30). 

2. A method according to Claim 1, characterized in that the number of pixels (n) 
10 selected for examination depends on the difference of pixel values across the block 

boundary (30). 

3. A method according to Claim 2, characterized in that the difference of pixel 
values is represented by the value A = |r ( - \\l where V\ is the value of the pixel on 
the first side of the block boundary closest to the boundary, and 1] is the value of the 

15 pixel on the second side of the block boundary closest to the boundary. 

4. A method according to Claim 2, characterized in that the number of pixels 
selected for examination also depends on the size of the quantization step of the 
transformation coefficients used in the transformation coding of the blocks. 

5. A method according to Claim 4, characterized in that the number of pixels 
20 selected for examination is determined by the formula 

0 A > 2.00a 

1 1.50a < A < 2.00a 



n - 



2 1 . 00a < A 

3 0.66a < A 

4 0.40a < A 

5 0.25a < A 

6 0 < A 



< 1.50a 

< 1.00a 

< 0.66a 

< O40a 

< 0.25a 



(2) 



wherein a = QPlog(QP) and QP is .the size of the quantization step of the 
transformation coefficients used in the transformation coding of the blocks. 



25 



6. A method according to Claim 1, characterized in that certain pixels to be 
filtered are selected from the pixels selected for examination, and a new value is 
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determined for each pixel to be filtered on the basis of pixels that appear in the 
filtering window set around the pixel. 

7. A method according to Claim 6, characterized in that the new value of the 
pixel to be filtered is the mean value of the pixels that appear in the filtering 

5 window. 

8. A method according to Claim 6, characterized in that for determining a new 
value for the pixels to be filtered on the first side of the block boundary, a filtering 
window is used, and the size of the window is determined by the table 



d r (d,> 1) 


1Y 


r 2 




1 


X 


X 


X 


2 


1 


X 


X 


3 


1 


1 


X 


4 


2 


2 


X 


5 


2 


2 


2 


6 


3 or 2 


3 


3 



10 wherein an integer parameter d r indicates activity on the first side of the block 
boundary, and an integer parameter d t indicates activity on the second side of the 
block boundary, r u r 2 and r 3 are the three pixels on the first side of the block 
boundary closest to the boundary in this order, X means that the pixel is not filtered, 
the number means that in addition to the pixel to be filtered, a quantity of pixels 

15 shown by the number are taken to the filtering window from both sides of the pixel 
to be filtered, and M 3 or 2 H means "3, if d, > 2, otherwise 2", and for determining the 
new value of the pixels to be filtered on the other side of the block boundary, a 
filtering window defined similarly is used, with the exception that all r's art- 
replaced by Ts and vice versa. 

20 9. A method according to Claim 8, characterized in that 

d r - 6, if |n - rj| < p/j with all je [ 1,6], 
otherwise: d r = i, where i must meet the conditions 
ie[l,n], 

|r f - r i+i | > p/i, and 
25 |r xjl < p/j with all je[I,i], 
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wherein the auxiliary parameter P = 41og(QP) and QP is the size of the quantization 
step of the transformation coefficients used in transformation coding of the blocks, 
and the value of the parameter d| is determined similarly, with the exception that all 
r's are replaced by l's. 

10. The use of the method according to claim 1 in a digital television receiver. 

11. A device for removing blocking artefacts from the frame of a digital video 
signal, which has been coded by blocks and then decoded, which device comprises 
a filter for correcting the value of pixels selected for examination, characterized in 
that it comprises means (41) that operate adaptively according to the image content 
of the frame, for selecting pixels for examination and filtering. 

12. A device according to Claim 11, characterized in that it comprises 
programmable means (41) for selecting pixels from the saved frame as the pixels to 
be examined, programmable means (44) for selecting the pixels to be filtered from 
among the pixels to be examined, and programmable means (43) for determining the 
new value of the pixels to be filtered. 
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